Papers with Dense representations
Extracting Text Representations for Terms and Phrases in Technical Domains (2023.acl-industry)
Copied to clipboard
| Challenge: | Large pre-trained language models are extensively used in modern NLP systems. |
| Approach: | They propose an unsupervised approach to encoding using character-based models and pre-trained sentence encoders to reconstruct large pre-trained embedding matrices. |
| Outcome: | The proposed approach matches the quality of sentence encoders in technical domains and is 5 times smaller and up to 10 times faster on high-end GPUs. |